Systems and methods for providing mobile proving ground

ABSTRACT

The disclosed embodiments include methods, systems, and articles of manufacture for enabling beta testing of new features within a native application. In one embodiment, a native application executed by a client device may include executable instructions that are activated and executed only upon indication of a user belonging to a beta test group. The client device may receive a signal from a service or content provider that includes test group information indicating whether a user is a member of a test group. The client device may process the received signal to determine whether the user is a member of a test group and, if so, activates and executes certain instructions received as part of the native application that define a user interface enabling access to additional functionality unavailable to a user determined not to be a member of a test group.

PRIORITY CLAIM

This application is a continuation of U.S. Application Ser. No.14/324,639, filed Jul. 7, 2014, which claims priority under 35 U.S.C. §119 to U.S. Provisional Patent Application No. 61/843,715, filed on Jul.8, 2013, the contents of both of which are expressly incorporated hereinby reference in its entirety their entireties.

TECHNICAL FIELD

The present disclosure relates generally to systems and methods formobile proving grounds. More particularly, and without limitation, thepresent disclosure relates to systems and methods for providing a userinterface in a mobile application that allows a set of users to test, inreal-time, new or modified features and functionality of the mobileapplication within the native mobile application.

BACKGROUND

The prevalence of mobile technologies such as smart phones, tablets, andother electronic mobile wireless devices has driven the proliferation ofmobile applications (or “apps”) that enable diverse functionality ofthese devices. As the use of smart phones, tablets, and other electronicmobile devices becomes even more widespread, it becomes increasinglyimportant to provide a convenient and enriching experience to clientsand consumers of businesses, institutions, and other content or serviceproviders (“providers”) through a well-designed app. For example, usersmay opt to transact with one provider over another based on theconvenience and functionality of that provider's app. With respect to afinancial services provider, the particular functionality of a mobileapp can eliminate the need for some users to visit a brick-and-mortarlocation for routine transactions.

As users continue to increasingly rely on the functionality of mobileapplications for their service and content needs, providers arechallenged with identifying the usefulness and desirability ofparticular features to incorporate into their mobile app. Additionally,providers strive to identify how users interact with the mobile app andto what extent additional or modified functionality can lead toincreased revenues from existing clients and/or attract new clients. Inparticular, providers desire a way to efficiently and effectively testthe use and desirability of new features in a mobile app by evaluatingreal-time interactions of a large number of users according to theiractual typical use of the mobile app.

Conventional testing methods include traditional surveying actual orpotential users of a mobile app regarding the desirability of particularfunctionality and how that functionality might be used. Traditionalsurveys, however, may not be very accurate in predicting how users willutilize that feature within the actual working (or “native”) mobile appthat is available for all users. Thus, real-time beta testing of anative app with the new features and functionality is more desirable.Conventional beta testing methods, however, are incapable of effectivelytesting the new features and functionality among a large group of actualusers. A provider typically must provide a beta version of its mobileapp (a “beta app”) to a mobile app distributor, such as the Apple AppStore® or Google Play®, where select users may be instructed to downloadthe beta app. But mobile app distributors often limit the number ofversions of a particular app that are available for download.Accordingly, a provider typically must include all new features orfunctionality it is testing within the beta app. Mobile app distributersalso limit the number of users that can download the beta app. Thus,conventional beta testing of new mobile app functionality typicallysuffers from being unable to release multiple versions of a beta app totest specific new features and functionality or to reach large numbersof users that may have diverse demographics and varying levels ofinteraction with the mobile app.

Typical beta testing methods also fail because they require select usersto download a beta version of the mobile app for each newly plannedfeature, which many of these select users may neglect or be reluctant todo. Additionally, some users may find that the beta app includes newfunctionality that is undesirable or results in a negative experience,requiring that user to once again download another version of the mobileapp, such as the native app. Thus, it is desirable to enable betatesting of new features for select users within the native app.Additionally, it is desirable to implement a method and system todynamically control the availability of new discreet features (betafeatures) of the mobile app to select users, without requiring separatedownloading of new versions of the mobile app for each beta feature.

SUMMARY

Disclosed embodiments include methods, systems, and computer-readablemedia configured to, for example, provide a user interface in a nativemobile application to a set of users for testing new features andfunctionality of the mobile application. For example, a mobile device isdisclosed for providing a user interface on a mobile application,comprising at least one processor and at least one memory device thatstores a set of instructions corresponding to the mobile application.When executed by the processor, the stored instructions may cause theprocessor to receive a signal from a service provider, the signalincluding information indicating test group membership data. The mobiledevice may also be configured to determine, based on at least thereceived signal, whether the user is a member of a test group and enableaccess to a first set of functions in the mobile application after theuser is determined to be one of the members of the test group. The firstset of functions may unavailable to non-members of the test group. Themobile device may also be configured to display a user interfaceincluding at least one feature from the first set of functions on themobile application.

In one aspect, the disclosed embodiments include a method for providinga user interface on a device. The method may include receiving acommunication signal including at least test group identificationinformation for a user associated with the device. The method may alsoinclude identifying whether the test group identification informationindicates the user is a member of at least one test group, and executinga first set of instructions after the user is identified as a member ofa test group associated with the first set of functions, and a secondset of instructions when the user is identified as not a member of thetest group. The method may also include displaying a user interfaceaccording to the execution of the first or second set of instructions,wherein the first set of instructions enable access to at least one ormore test functions that are un-available to a user that is not a memberof a test group.

Aspects of the disclosed embodiments may include tangiblecomputer-readable media that stores software instructions that, whenexecuted by one or more processors, are configured to and capable ofperforming and executing one or more of the methods, operations, and thelike consistent with the disclosed embodiments. Also, aspects of thedisclosed embodiments may be performed by one or more processors, thatare configured as special-purpose processor(s) based on softwareinstructions that are programmed with logic and instructions thatperform, when executed, one or more operations consistent with thedisclosed embodiments.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the disclosed embodiments, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate disclosed embodiments and,together with the description, serve to explain the disclosedembodiments. In the drawings:

FIG. 1 is a block diagram of an exemplary system, consistent withdisclosed embodiments.

FIG. 2 is a block diagram of an exemplary client device system,consistent with disclosed embodiments.

FIG. 3 is a block diagram of an exemplary application system, consistentwith disclosed embodiments.

FIG. 4 is a flowchart of an exemplary application download process,consistent with disclosed embodiments.

FIG. 5 is a flowchart of an exemplary beta user registration process,consistent with disclosed embodiments.

FIG. 6 is a flowchart of an exemplary client device application process,consistent with disclosed embodiments.

FIG. 7 is a flowchart of an exemplary beta user identification process,consistent with disclosed embodiments.

FIGS. 8A-8D are illustrations of exemplary user interfaces, consistentwith disclosed embodiments.

FIGS. 9A and 9B are illustrations of an exemplary rating interface,consistent with disclosed embodiments.

DETAILED DESCRIPTION

Reference will now be made in detail to the disclosed embodiments,examples of which are illustrated in the accompanying drawings. Whereverconvenient, the same reference numbers will be used throughout thedrawings to refer to the same or like parts.

FIG. 1 is a block diagram of an exemplary system 100 for performing oneor more operations consistent with the disclosed embodiments. As furtherdisclosed herein, system 100 may be used for providing a user interfacein a native mobile application to a set of users for testing newfeatures and functionality of the mobile application. In one embodiment,system 100 may include one or more service or content applicationsystem(s) 112 (“application system”) interconnected via a network 120 toone or more client devices 130. Application system 112 may be one ofmany systems controlled, operated, or utilized by a service or contentprovider 110 (“provider”). In other embodiments, application system 112is a separate and distinct system from service or content provider 110.System 100 may include other components or subsystems that perform orassist in the performance of one or more processes consistent with thedisclosed embodiments as would be understood by one of ordinary skill inthe art.

As will be described in more detail below, in certain embodiments, oneor more components of system 100 may be configured to select one ormore, or a subset of, users 132 to join a beta test group for testingnew features and functionality of a mobile application. Such selectusers may be designated as beta users 134. System 100 may notify thebeta users 134 of the availability to join the beta test group. In someembodiments, system 100 may notify the beta users 134 via, for example,client device 130, personal computer, telephone, traditional mail, orother communication medium. System 100 may receive enrollmentinformation from at least one beta user 134 for enrolling in the betatest group and may store that information or otherwise updateinformation in a database related to the beta testing of new or updatedmobile application features. System 100 may enable users 132 and 134 todownload and execute a native mobile application on client devices 130.Upon execution of the application, system 100 may receive userinformation from a user (via, e.g., client device 130) and determinewhether the user is one of beta users 134. System 100 may then transmitinformation to client device 130 indicating whether the user is one ofbeta users 134 via an application signal, e.g. a signal including asoftware command, program variable setting, etc. Client device 130 maydetermine whether the signal information indicates that the user is oneof beta users 134. If the user executing the application is one of betausers 134, client device 130 may display one or more beta features fortesting on a user interface of the native application. If the userexecuting the application is not one of beta users 134, i.e., user 132,client device 130 may display the normal or current features of thenative mobile application.

In an exemplary embodiment, provider 110 corresponds to a business,merchant, or entity that provides goods, services, and/or informationcontent, such as a retailer, grocery store, service provider (e.g.,utility company, etc.), non-profit organization, content provider(whether offline or online), or any other type of entity that providesgoods, services, and/or information that consumers (i.e., end-users orother business entitles) may purchase, consume, use, etc. Provider 110is not limited to entities that conduct business in any particularindustry or field.

Provider 110 may be associated with a merchant brick-and-mortarlocation(s) that a consumer (e.g., users 132, 134) may visit in personand purchase and/or receive goods and services. Such physical locationsmay include computing devices that perform financial servicetransactions with consumers (e.g., Point-of-Sale (POS) terminal(s),kiosks, etc.). According to some embodiments, provider 110 may performfinancial transactions, e.g., purchase transactions, with users 132, 134(via, e.g., client device 130 operate by users 132, 134). Provider 110may also include back and/or front-end computing components that storedata and execute software instructions to perform operations consistentwith disclosed embodiments, independent of or in conjunction withapplication system 112.

Provider 110 may also be associated with a merchant that provides goodsor services via online or e-commerce solutions. For example, provider110 may be a merchant selling goods via a website using known online ore-commerce systems and solutions to market, sell and process onlinetransactions. Provider 110 may also provide such services through amobile application, such as a native mobile app operating on clientdevice 130, which may communicate with application system 112 vianetwork 120.

In an exemplary embodiment, provider 110 may correspond to a financialservices provider. For example, the financial services provider may beassociated with a bank, stock broker, fund manager, credit card issuer,or other type of financial services entity or institution thatgenerates, provides, manages, and/or maintains financial serviceaccounts for one or more users. Financial service accounts may include,for example, credit card accounts, loan accounts, checking accounts,savings accounts, stock accounts, reward or loyalty program accounts,and/or any other type of financial service account known to thoseskilled in the art. The financial services provider is capable ofproviding such services through a mobile application, such as a nativemobile app operating on client device 130 that communicates withapplication system 112 via network 120. In this embodiment, financialservice provider 110 may also include one or more physical locations andthe infrastructure and systems configured to generate and/or provide thedesired financial services.

In another embodiment, provider 110 may be a system associated with anentity providing content, which is viewed, listened to, or otherwiseconsumed by users. For example, provider 110 may be associated with abroadcasting network, radio network, website, news service, socialnetwork, digital media source, or other type of entity that generates,provides, manages, distributes, and/or delivers content (whether offlineor online) to others. For instance, as non-limiting examples, provider110 may be associated with a television network, cable company, radioservice, news service, or digital media source. Provider 110 may deliverto client device 130, via network 120, dynamic media content (i.e.,video, audio, interactive content, etc.) and/or static content (digitalprint media, images, etc.). The content delivered to client device 130may be provided to a user 132, 134 through an application operating onthe client device, such as a native application or a web browseroperating on client device 130, which communicates with applicationsystem 112 via network 120. Provider 110 may further include additionalinfrastructure and systems configured to generate, provide, manage,distribute, and/or deliver content. As noted above, however, provider110 is not limited to entities that conduct business in any particularindustry or field.

Application system 112 may include a number of computing systemsconfigured to provide services or content to users 132,134 via network120 and client device 130 according to the disclosed embodiments. Asfurther described herein, components of application system 112 mayinclude one or more computing devices (e.g., computer(s), server(s),processor(s) etc.), memory systems for storing data and content, and/orsoftware instructions (e.g., database(s), memory devices, etc.), andother known computing components. In some embodiments, the one or morecomputing devices are configured to execute software instructions storedon one or more memory devices to perform one or more operationsconsistent with the disclosed embodiments. Although computing devicesoperating according to disclosed embodiments may be implemented ascomputer processing instructions, all or a portion of the functionalityof the computing devices may be implemented instead in electronicshardware. The components of application system 112 may be provided in asingle location or distributed across a number of locationsinterconnected via one or more networks. Additionally, applicationsystem 112 may be directly or indirectly controlled or operated byprovider 110, or may be part of a third party system on behalf ofprovider 110. In some embodiments, one or more aspects or components ofapplication system 112 may be operated or controlled by provider 110,whereas other aspects and components are operated or controlled by athird party.

Network 120 may be any type of network configured to providecommunications between components of system 100. For example, network120 may be any type of network or combination of networks that providescommunications, exchanges information, and/or facilitates the exchangeof information, such as the Internet, a Local Area Network, a Wide AreaNetwork, a wired or wireless network, a cellular network, or othersuitable connection(s) and infrastructure that enables the sending andreceiving of information between the components of system 100. Thedisclosed embodiments are not limited to any particular configuration ofnetwork 120.

Client device(s) 130 may be one or more computing devices configured toperform one or more operations consistent with the disclosedembodiments. In an exemplary embodiment, client device 130 may be amobile device (e.g., tablet, smartphone, personal digital assistant(PDA), e-reader, iPad®, iPod®, etc.). Disclosed embodiments, however,are not limited to any particular configuration of client device 130. Assuch, client device 130 may be a desktop computer, a laptop, smarttelevision or any other type of computing device or any other devicecapable of presenting content or displaying a user interface to user 132for receiving a service or content via network 120. According to someembodiments, client device 130 may comprise a network-enabled computingdevice operably connected to one or more other presentation devices,which may themselves constitute client devices 130.

As shown in FIG. 2, client device 130 may include one or morecontrollers or processors 220, which may include one or moremicroprocessors configured to control the operation of client device 130according to the disclosed embodiments. Processor 220 may be configuredto interact with a number of device subsystems, such as display 210,input device 212, communication unit 214, and memory unit 230. Clientdevice 130 is not limited to the particular components or subsystemsillustrated in FIG. 2, and may include any number of additionalsubsystems or arrangements of subsystems for performing the operationsof client device 130 consistent with the disclosed embodiments.

In an exemplary embodiment, display 210 may comprise a screen fordisplaying content or providing a user interface. Display 210 mayinclude a visible screen realized by underlying LED-backlit LCDtechnology, organic LED technology, electronic ink, or any othersuitable display technology according to the disclosed embodiments. Insome embodiments, display 210 may be an external component incommunication with client device 130 for displaying content or a userinterface consistent with the disclosed embodiments. Input device 212may incorporate a keyboard realized using hardware or software, one ormore buttons, a stylus, a trackball or any other input device forenabling input or control by user 132. In some embodiments, input device212 may be provided as part of display 210, such as through atouch-sensitive input surface or touch-screen or any other method ofrecognizing user input via display 210.

Communication unit 214 may include one or more communication systemssuitable for communicating via network 120. The particular design ofcommunication unit 214 depends on the configuration of network 120,through which client device 130 is capable of communicating withapplication system 112. In the disclosed embodiments, client device 130is capable of sending and receiving communication signals over aplurality and/or combination of network configurations, such as wired,wireless, cellular, radio, etc.

Memory unit 230 may store software instructions which may include one ormore operating systems (“OS”) 232, one or more applications (“apps”)234, and data 236. Processor 220 is configured to execute softwareinstructions stored in memory unit 230 to communicate via network 120and to carry out a number of operation processes and applications torealize functionality of client device 130. For example, client device130 may execute software instructions that may be provided as a nativeapplication or app 234 to generate and display a user interface and/orcontent via display 210, which may be internal to or operably connectedto client device 130. Applications 234 may enable not only the displayof content and/or a user interface but may also enable client device 130to communicate with application system 112 (FIG. 1) in order to receiveadditional content from and/or the provision of services with provider110 (FIG. 1), such as mobile or online banking. Applications 234 mayinclude software instructions for a wide range and number ofapplications or apps configured to run on operating system 232,including a native application associated with provider 110, an internetbrowser application, and/or a content player application. Operatingsystem 232 includes one or more software instructions according to anyplatform suitable for controlling operation of client device 130according to the disclosed embodiments.

In the disclosed embodiments, one or more of applications 234 may berealized as a native application, a web application, or a combinationthereof. With respect to a native application, the software code may bedeveloped for use on a particular client device 130 or according to aparticular platform or operating system 232 controlling thefunctionality of client device 130. In an exemplary embodiment, a nativeapplication may be installed directly onto client device 130 and storedin memory unit 230. The native application may be retrieved ordownloaded from an application distributor or directly from applicationsystem 112. In an exemplary embodiment, data and content associated withthe native application may be stored in memory unit 230 as data 236. Aweb application, on the other hand, may be generalized for a number ofplatforms and may enable receipt of content that is stored atapplication system 112 via network 120 as opposed to being storedlocally on client device 130. Web applications may be accessed using aseparate web browser application and/or other native application storedon client device 130, which includes instructions for retrieving contentfrom application system 112 via network 120. Applications 234 accordingto exemplary embodiments are not restricted to any particularconfiguration or label applied to the application, and are describedherein according to their functionality.

FIG. 3 shows an exemplary application system 112, a subsystem ofprovider 110, for implementing embodiments consistent with the presentdisclosure. Variations of application system 112 may be realizedaccording to a particular provider 110. As shown in FIG. 3, in oneembodiment, application system 112 may include a content server 320, anauthentication server 330, and a database 340. According to someembodiments, content server 320 may comprise one or more web servers orsimilar computing devices that generate, maintain, and provide websitesor web content consistent with disclosed embodiments. Content server 320and authentication server 330 may further include one or moreprocessors, one or more memories, and one or more interfaces forcommunicating with database 340 and network 120 (FIG. 1) and/or eachother. Content server 320 and authentication server 330 may be anycomputing device capable of performing the desired functionality, suchas a general purpose computer, a mainframe computer, or any combinationof these components. Alternatively, content server 320 andauthentication server 330 may be configured as a particular apparatus,embedded system, dedicated circuit, and the like based on the storage,execution, and/or implementation of the software instructions thatperform one or more operations consistent with the disclosedembodiments.

Content server 320 and authentication server 330 may be standaloneserver units, or they may be part of a subsystem, which may be part ofan even larger system. Each of the shown components need not be providedwithin a single location and may be widely distributed among variousother systems or subsystems. For example, application system 112 mayinclude distributed servers that are remotely located and communicateover a network (e.g., network 120), which may include a dedicatednetwork such as a LAN. In some embodiments, the functionality of contentserver 320 and authentication server 330 may be encompassed in a singleserver or server system.

Content server 320 and authentication server 330 may also becommunicatively connected to one or more database(s) 340. Database 340may include one or more memory device(s) that store information, whichmay be accessed and/or managed through content server 320,authentication server 330, and/or other systems provided by provider110. By way of example, database 340 may include Oracle™ databases,Sybase™ databases, or other relational databases or non-relationaldatabases, such as Hadoop sequence files, HBase, or Cassandra. Databases340 may include, for example, data and information related to the sourceand destination of a network request, multimedia content, web pagecontent, user information etc. according to the disclosed embodiments.In some embodiments, database 340 may be located remotely fromapplication system 112. Database 340 may include computing components(e.g., database management system, database server, etc.) configured toreceive and process requests for data stored in memory devices ofdatabase 340 and to provide data from database 340.

As shown in FIG. 4, according to an exemplary embodiment, a nativemobile application associated with provider 110 (FIG. 1) may beinstalled onto a client device 130 via application download process 400.In an exemplary embodiment, application system 112 may receive a requestfrom client device 130 to download a native application to client device130 (step 410). In one embodiment, application system 112 may receivethe request from an application distributor navigated by user 132, 134using a web browser application installed on client device 130 or anyparticular application installed on client device 130. In anotherembodiment, application system 112 may receive the request to download anative application associated with provider 110 onto client device 130from a webpage or some other portal associated with a provider 110accessed by user 132, 134 via, e.g., client device 130. In thisembodiment, application system 112 may store software instructionscorresponding to a native application in content server 320 and/ordatabase 340. In one embodiment, application system 112 may receiveadditional information from client device 130 regarding the particulardevice specifications of client device 130 to enable client device 130to download software instructions corresponding to its particularspecifications. In yet another embodiment, application system 112 maypush a download request link to client device 130 or transmit thesoftware code corresponding to a native application directly to clientdevice 130 in, for example, an e-mail, a text or short message service(“SMS”) message, a prompt through an app, or other suitable method. Instep 420, client device 130 may receive the native application softwarecode, preferably via network 120, and downloads and installs thesoftware code to be stored in memory unit 230 as one of applications 234(step 430).

In an exemplary embodiment, provider 110 includes a financial servicesprovider, and the native application, once executed by processor 220 inclient device 130, provides a user interface (such as shown in FIG.8A-D) for receiving various financial services information and forperforming various transactions with financial services provider 110. Inone embodiment, a native application may include software code definingthe content and placement of particular fields (e.g., text fields, inputfields, graphic fields, icons, colors, etc.) and directing client device130 to display a user interface of the native application according tothe software code. For example, the native application software codeincludes instructions for generating a “standard” (i.e. non-beta) menuscreen 810 and each of the displayed menu features and options as shownin FIG. 8A. In one embodiment, the native application software code alsoincludes instructions to display additional information or graphics orto execute a process in response to a selection received from user 132of any one of the menu features displayed. For example, one such processmay include accessing, via network 120, additional information stored orgenerated by application system 112. Such process performed in responseto receiving user 132's, 134's selection may include receiving personalfinancial information, such as the user's account balance fromapplication system 112, and displaying such information within thenative application interface. Therefore, according to the disclosedembodiments, the native application software code enables client device130 to access to personal financial information stored on client device130 and information retrieved remotely, such as from application system112.

As shown in FIGS. 8A and 8B, an exemplary native application for afinancial services provider 110, such as one of applications 234executed on client device 130, may provide a user interface displaying anumber of convenient features for improving a user's access to financialservices. For example, as shown in menu 810, a user 132, 134 may accessinformation concerning a particular account, such as a savings accountoffered by Capital One 360®, by selecting the “My Accounts” field 812.Additionally, a user 132, 134 may access another account, such as acredit card account or a checking account by selecting the “CreditCards” field 816 or the “Capital One Bank” field 818, respectively. Anumber of other features or functionality may be realized as suggestedby the remaining menu items. For example, the native application mayenable user 132, 134 to transfer money by selecting field 813, depositchecks by selecting field 814, pay a bill by selecting field 815, etc.The particular features shown are provided only as an example, and arenot limiting of any of the disclosed embodiments. Additionally, thedisclosed embodiments are not limited to a particular design of a nativeapplication user interface. As such, the functionality of a nativeapplication may be realized by a user interface displaying one or moreicons relating to particular features, or any other method desired byprovider 110.

According to an exemplary embodiment, each user 132,134 operating clientdevice 130 may receive native application software code associated witha particular provider 110. Application system 112 may provide a singleversion of a native application available for download by any user 132,134. While the visual appearance of the native application userinterface may vary for some users 132, 134 depending on thespecifications of client device 130, in one embodiment, a particularnative application may contain substantially the same functionality foreach client device 130, to the extent such functionality is not limitedby the specifications of client device 130. In some embodiments,application system 112 may provide multiple versions of a nativeapplication, one for each of varying specifications for client devices130. For example, one version of a native application may be providedfor similarly functional smartphones, and another version of the nativeapplication may be provided for a smart television, and yet anotherversion for a tablet.

The present disclosure provides systems, methods, and computer readablemedia for improving the delivery of content or the provisions ofservices to users 132, 134 when accessing and using a native applicationon client device 130. The functionality of a provider's 110 nativeapplication or app may be a significant factor toward increasing thetotal client base, thereby increasing revenue. As such, the presentdisclosure describes methods and systems to develop and test newfunctionality and new features (“beta features”) for the nativeapplication to keep a user engaged with the app, as well as to increasethe usability and convenience of the app. According to the presentdisclosure, provider 110 is enabled to test the real-time functionality,use, and desirability of beta features for a future version of thenative app within an enrolled or current version of the nativeapplication before making the beta features available to all users 132of the native application.

In FIG. 5, a method 500 is provided for identifying and enrolling a betauser 134 as a user of a beta test group according to some embodiments.In step 505, application system 112 identifies a select group of betausers 134 to test beta features of the native application. In oneembodiment, the select group of beta users 134 is a subset of all users132. The beta users 134 may be selected from a current group ofcustomers of provider 110, or current users 132 of the nativeapplication provided by application system 112.

In one embodiment, application system 112 may select the beta users 134according to any number of criteria. For example, the beta users 134 maybe chosen from the most frequent users of a current version of thenative application, or may be chosen based on particular demographicssuch as age, geographic location, net value of accounts, duration ofbeing a customer, etc. In another embodiment, the beta users 134 may bedetermined from a random sample, or may be selected as a diverse groupwith little or no identifying characteristics. In yet anotherembodiment, application system 112 may seek interest from the generalpool of users 132 to become one of beta users 134 without identifyingany particular group of users. In the disclosed embodiments, a beta testgroup may include several hundred beta users 134 to several thousandbeta users 134. A beta test group may, alternatively, contain more orless beta users 134. There is no artificial minimum or maximum on thenumber of beta users 134 that may be chosen to participate in a betatest group.

Application system 112 may notify the selected users of their ability tojoin a beta test group (step 510) according to any effective method forproviding notices or communications to select users. For example,application system 112 may communicate with prospective beta users 134via e-mail or text message, or upon the selected user accessing theiraccount associated with the provider 110, either online or through anative application of application system 112 via client device 130. Inanother embodiment, a link may be provided to the prospective beta users134, directing them to a webpage or portal or the like, which enablesthe beta user 134 to participate in an enrollment or pre-registrationprocess to join a beta test group. In another embodiment, theprospective beta users 134 may be requested to reply to any particularmethod of communication to enroll in a beta test group. Once applicationsystem 112 receives a user agreement to enroll in a beta test group(step 520), via, e.g., client device 130, application system 112 mayreceive user information from the enrolling beta user 134 (step 530).User information may include a username, login identifier, accountnumber, or any other credential or identifier used by application system112 to identify a user 132,134. In another embodiment, applicationsystem 112 may gather user information regarding the enrolling beta user134 from information stored, for example, in database 340. Afterobtaining information concerning the enrolling beta user 134,application system 112 may update beta test group data stored, forexample, in database 340 to include the enrolling beta user's 134information (step 540). Application system 112 may store a record orlog, such as in database 340 or in authentication server 330 (FIG. 3),including identifying information for each beta user 134 enrolled in abeta test group. In another embodiment, application system 112 mayupdate user identification records stored in database 340 with, forexample, a flag or other indicator identifying the beta user 134 asenrolled in one or more beta test groups.

In another embodiment, steps 510 and 520 of beta user registrationprocess 500 may be omitted. For example, application system 112 mayautomatically enroll one or more of select users identified in step 505in a beta test group. In this embodiment, application system 112 mayenroll select beta users 134 without notifying the user. Applicationsystem 112 may gather user information (step 530) regarding the selectbeta users 134 from information previously received or stored byapplication system 112.

According to one embodiment, a single beta test group may be chosen totest all available beta features of a native application. As such, everybeta user 134 may be enabled to test all beta features planned for thenative application. In another embodiment, there may be more than onebeta test group. For example, there may be a first beta test group fortesting a first beta feature of the native application, and a secondbeta test group for testing a second and different beta feature of thenative application. Additionally, there may be one or more combinationsof beta test groups associated with one or more beta features or groupsof beta features. In the exemplary embodiments, application system 112may store identifying information for each beta user 134, including anindication of one or more beta test groups in which the beta user 134 isselected to participate. In some embodiments, identification informationfor each beta user 134 may correspond to information associated withclient device 130, such as a device serial number, a network identifier,or any other indicia for identifying client device 130.

Once provider 110 or application system 112 enrolls a beta user 134 in abeta test group according to registration process 500, for example, thatbeta user 134 may be enabled to access new beta features of an exemplarynative application according to application process 600 (FIG. 6)executed by client device 130. Prior to beginning application process600, a beta user 134 may have completed the application download process400. The latest, most recent, or most current version of the nativeapplication software may have been installed onto client device 130.According to an exemplary embodiment, the version of the nativeapplication software that has been installed onto client device 130 of abeta user 134 may be the same version installed on client devices ofother users 132 that are not part of a beta test group. According tothis embodiment, beta features may be made available to beta users 134within the native application. As such, a beta user 134 may not berequired to download and install a different version or a separate, betaversion of the exemplary native application specifically to test thebeta features. As discussed in detail below, the native applicationsoftware provided to all users 132 may include “hidden” code definingthe beta features or functionality and defining an interface to accessthe beta features. The “hidden” code may be “activated” or executed byclient device 130 upon receipt of a signal from application system 112,indicating that the particular user associated with client device 130 isa member of a beta test group (i.e., a beta user 134).

FIG. 6 shows an exemplary application process 600 according to thedisclosed embodiments. According to process 600, a user 132 or 134 mayexecute a native application via client device 130 (step 610). A user132 or 134 may execute the native application by selecting an iconcorresponding to the native application by any input means available forclient device 130 or by any other method. As discussed in detail above,an exemplary native application may be one of several applications 234installed in memory unit 230 of client device 130. Upon execution of thenative application, processor 220 (FIG. 2) may execute nativeapplication software to display a user interface via display 210. In anexemplary embodiment, the native application may display a user loginscreen requesting login credentials or other identifying informationfrom user 132 or 134 for authenticating the user to receive specificcontent or services. In such an embodiment, user 132 or 134 may enter orotherwise provide the requested login information (step 620), such asvia input device 212. In another embodiment, a native application mayretrieve user identification information, such as information stored asdata 236 in memory unit 230, without requiring user input. In yetanother embodiment, user identification information may correspond toidentification information associated with client device 130 as opposedto user 132 or 134. In such an embodiment, the native application maynot require any user authentication procedure.

Upon receiving or retrieving user identification information (step 620),client device 130 executing the native application may transmit the userinformation to application system 112 (step 630) via network 120utilizing communication unit 214.

Referring now to FIG. 7, illustrating an exemplary beta useridentification process 700, application system 112 may receive thetransmitted user identification information (step 710) from clientdevice 130 via network 120. Application system 112, via authenticationserver 330, for example, may determine whether the user 132,134 isauthorized to access content or receive a service (step 720). In oneembodiment, the native application enables access, e.g., by clientdevice 130, to user-specific financial information stored at oraccessible by application system 112. In such an embodiment, applicationsystem 112 may determine whether the received user identificationinformation corresponds to an authorized user. For example,authentication server 330 may receive user identification informationand compare such information to other data stored by authenticationserver 330 or database 340. If authentication server 330 determines thatuser identification information does not correspond to an authorizeduser (step 720; NO), then application system 112 may transmit anun-authentication signal to client device 130 (step 722) indicating thatthe particular user 132, 134 is unable to be authenticated byapplication system 112. According to some embodiments, the nativeapplication does not require or desire user authentication, and step 720may be omitted.

If user authentication is required and user 132, 134 is authenticated(step 720; YES), application system 112 may determine whether the useris a beta user 134 (step 730), as discussed above. The determination instep 730 may be performed by authentication server 330, content server320, or any other appropriate means. In an exemplary embodiment,application system 112 may compare user identification informationreceived from client device 130 with a beta test group recordidentifying enrolled beta users 134. Additionally or alternatively,application system 112 may inspect a record corresponding to theidentified user and determine whether that user is “flagged” as a betauser 134 or whether the user record includes any indicia indicating thatuser is a beta user 134. As discussed above, in an alternativeembodiment where user identification information corresponds toidentification information of client device 130, determination as towhether the client device 130 is part of a beta test group may beperformed in a similar manner. In the exemplary embodiments, ifapplication system 112 has established more than one beta test group,application system 112 may determine each beta test group in which thebeta user 134 or client device 130 is indicated as being a member.

Application system 112 may also transmit a signal to client device 130,including information indicating whether the client device 130 or userbelongs to a beta test group. In an exemplary embodiment, the signaltransmitted by application system 112 may be according to thespecification of any network 120 or combinations of networks. If theuser or client device 130 is a member of a beta test group (step 730;YES), application system 112 transmits a beta test group signal toclient device including information indicating the one or more beta testgroups to which the beta user 134 or client device 130 belongs (step740). If a user 132 or client device 130 has not been previouslyenrolled as a beta test group member (step 730; NO), then applicationsystem 112 transmits a non-test group signal to client device (step750). In one embodiment, application system 112 may also includeauthentication information as part of the signal transmitted in step 740or 750, if authentication is required.

Returning back to FIG. 6, client device 130 may receive the signaltransmitted in step 740 or 750 via network 120 and communication unit214 (step 640). Processor 220 may be configured to process the receivedsignal and to perform a next process under the instructions of thenative application software and according to the received signal orsignals. In step 640, client device 130 may determine whether thereceived signal includes any beta test group information. If thereceived signal includes beta test group information (step 645; YES),the native application may display a user interface that enables betauser 134 to access one or more new beta features (step 650). Forexample, as shown in FIG. 8B, a beta test menu 811 a may be made visibleonly for beta users 134. In one embodiment, beta test menu 811 a mayinclude an additional menu item or field such as the “Feature Center”field 822 displayed under “Tools” heading 820. As compared to a menu 810in FIG. 8A that is displayed to non-beta users 132, beta test menu 811 ashown in FIG. 8B enables a beta user 134 to access beta features of thenative application by selecting the “Feature Center” field 822. Ifclient device 130 determines that the received signal from applicationsystem 112 does not include beta test group information (step 645; NO),client device 130 may execute the native application softwareinstructions for displaying a “standard” non-beta test menu 810 (step660) that does not include access to “Feature Center” field 822, forexample.

In another embodiment, beta test group information may be stored atclient device 130 once received from application system 112 according tothe above embodiments. Thus, after receiving a signal from applicationsystem 112 for the first time in step 640, client device 130 may storethe received beta test group information in memory 230, for example, tobe accessed each subsequent time a user 132 or 134 executes the nativeapplication. In another embodiment, a native application may alsoinclude software instructions to modify itself upon receipt of a firstsignal from application system 112 in step 640 to automatically displaya beta test group interface, such as beta test menu 811 a, upon eachsubsequent execution of the native application at client device 130.Thus, beta features may be made available to beta user 134 withoutreceiving beta test group information from application system 112 orwithout retrieving such information from memory unit 230, for example.

In yet another embodiment, steps 620, 630, and 640 may be omitted orperformed at another time, such as during the native applicationdownload process 400. For example, prior to or as part of requestingdownload of a native application (step 410), a user 132, 134, via, e.g.,client device 130, may provide identification information to applicationsystem 112. During that process, application system 112 may determinewhether the user and/or client device 130 is enrolled in a beta testgroup in a manner that may be similar to step 730 (FIG. 7). If it isdetermined that the user or client device 130 is enrolled in a beta testgroup, application system 112 may provide additional beta test groupinformation along with the native application software for installationat client device 130. As such, the installed native application mayinclude instructions for executing code for displaying a beta test menu811 a, such as shown in FIG. 8B, upon execution of the nativeapplication. Accordingly, in one embodiment, a beta user 134 may be ableto access beta features in the native application without specificallyreceiving a signal from application system 112 upon execution of thenative application.

In an alternative embodiment, upon application system 112 identifying auser as a beta user 134 (step 730; YES), application system 112 maytransmit or push to client device 130 additional code corresponding tobeta functionality instead of or together with beta test groupinformation (step 740). For example, application system 112 may transmitexecutable code or instructions that when processed by processor 220 mayenable beta functionality within the native application for a beta user134. The additional code may include instructions for displaying“Feature Center” field 822 or other beta features, as discussed below.

In some embodiments, as shown in FIGS. 8B and 8C, a beta user 134 may beenabled to access and test new beta features by selecting “FeatureCenter” field 822. Upon selecting “Feature Center” field 822, the nativeapplication may instruct processor 220 to display a subsequent menuscreen 830 corresponding to the Feature Center, as shown in FIG. 8C. Thecode for displaying the subsequent menu screen 830 may be provided orreceived as part of the native application. As shown in FIG. 8C, anumber of beta features such as features 836, 837, and 838 may be madeavailable to a beta user 134 for beta testing. Beta user 134 may thenselect a download/install icon 839 corresponding to each beta featuredisplayed to install that particular feature to client device 130. Inone embodiment, software code corresponding to a particular betafeature, such as feature 836, may be received by client device 130together with the native application during the download process, suchas during step 420 (FIG. 4). As such, each beta feature may already bestored at client device 130 prior to selecting download/install icon839. In an alternative embodiment, upon selecting download/install icon839 corresponding to a particular beta feature, beta user 134 may bedirected to subsequently receive and download software code for the betafeature from application system 112 in a manner similar to applicationdownload process 400.

According to the exemplary embodiments, the beta features may includeany particular functionality that provider 110 desires to provide users132, 134 of its native application. For example, as shown in FIG. 8C,one such beta feature may correspond to a “Refer a Friend” feature 836.In one embodiment, “Refer a Friend” feature 836 enables a user 132, 134of the native application to recommend the native application to anotherfriend. Beta features 836, 837, and 838 may enable new transactions oroperations, and in the disclosed embodiments, such beta features arefully workable within the native application.

In one embodiment, application system 112 can dynamically control whichbeta features within the Feature Center menu 830 are made available, ifat all, to select groups of beta users 134. For example, applicationsystem 112 may form a number of beta test groups for testing particularbeta features of the native application. In this embodiment, applicationsystem 112 in step 740 (FIG. 7) may include such group informationindicating the one or more beta test subgroups to which the beta user134 or client device 130 belongs. As such, the native applicationexecuted at client device 130 may execute specific software codedirected only to the one or more beta features designated for aparticular beta test group according to the received beta test groupinformation. For example, a particular beta user may not be able toaccess or view every beta feature contemplated by application system112. As such, upon accessing the Feature Center shown in screen 830(FIG. 8C), only select beta features may be made available for aparticular beta user 134 to install on client device 130 according toreceived beta test group information.

In another embodiment, application system 112 may dynamically update oradd new beta features accessible via a Feature Center menu 830 for anyparticular beta test group. According to this embodiment, Feature Centermenu 830 may provide access to beta features stored at applicationsystem 112, for example, via one or more links to the beta features. Inone embodiment, the one or more links may correspond to a storagelocation enabling a beta user 134 to receive and download software codefor the beta feature from application system 112. In another embodiment,the one or more links may direct a beta user 134 to a mobile web pageproviding access to one or more beta features, as discussed below.

In yet another embodiment, Feature Center menu 830 may be provided as amobile web page that is accessible within the native application. Assuch, a beta user 134 may be redirected within the native application toan internet web page, such as a mobile web page, upon selecting the“Feature Center” field 822. In this embodiment, the mobile web page maybe provided by content server 320 of application system 112. Applicationsystem 112 may then dynamically add or update any number of betafeatures which are made accessible to beta user 134 through such amobile web page. Beta user 134 may then be able to select (via, e.g.,client device 130) which of the new beta features it desires to installonto client device 130 according to the above embodiments. For example,upon selecting a download/install icon 839, client device 130 mayreceive software corresponding to the selected feature for downloadsimilar to application download process 400 (FIG. 4).

In another embodiment, all users 132, 134 of a native application mayhave access to the Feature Center menu 830 within the nativeapplication, however, only beta users 134 may be able to see and/orselect one or more beta features. For example, a Feature Centeraccording to the alternative embodiment may include a number offeatures, some of which are beta features made available only to betausers 134. In such an embodiment, the native application may directnon-beta users 132 to a first Feature Center displaying featuresavailable for all users, whereas beta users 134 may be directed to asecond Feature Center displaying additional beta features. The separatefirst and second Feature Centers may be individual mobile web pagesaccessible within the native application, similar to the aboveembodiment. In this embodiment, the native application may direct a betauser 134 to the second Feature Center web page according to beta testgroup information received from application center 112, such asaccording to steps 640 and 740 (FIGS. 6 and 7, respectively).

In yet another embodiment, application system 112 may transmit or pushadditional software code or instructions corresponding to new betafeatures to client device 130. Application system 112 may push theadditional code to a select group of beta users 134 or all beta users134. The additional beta feature code may be transmitted to clientdevice 130 of a beta user 134 upon executing the native application atclient device 130. Alternatively, application system 112 may transmitnew beta feature code to beta users 134 on a scheduled release or at anydesired time for releasing new beta functionality to beat users 134. Forexample, application system 112 may transmit executable code orinstructions that when processed by processor 220 enable a beta user 134to execute the beta features and functionality. The additional code mayinclude instructions for displaying the beta feature in beta featurescreen 830, as shown in FIG. 8C, corresponding to the Feature Center.The additional code may also include instructions enabling functionalityof the new beta feature upon installation of the beta feature in themanner discussed above.

Upon installing a beta feature, such as 836, 837, or 838, at clientdevice 130, according to any of the above embodiments, a nativeapplication may be configured to display the installed beta featurewithin a user interface such as shown in menu 811 b in FIG. 8D.Interface menu 811 b may be substantially the same as the beta test menu811 a (FIG. 8B), except it may provide one or more new fields within thenative application interface corresponding to the installed one or morebeta features, such as field 824 corresponding to an installed “Refer aFriend” beta feature 836, by way of example. The exemplary embodimentsare advantageous at least because a beta feature can be tested inreal-time, or substantially real-time, as part of an interface as wouldbe implemented in a full rollout of the feature within a nativeapplication made available for all users 132,134. For example, a betauser 134 can now access and test “Refer a Friend” beta feature 836 as itwould any other feature available in the standard version of the nativeapplication available for all other users 132. Furthermore, beta user134 is not required to download a new beta version of the nativeapplication in order to test a beta feature. Additionally, beta user 134is enabled to choose to install only those beta features that it mayfind desirable and uninstall any beta feature it does not desire, suchas by selecting an uninstall icon 904 shown in FIG. 9A. Accordingly, theoverall beta test experience for a beta user 134 is more closelysimulated to a regular user experience and is more convenient for thebeta user 134, thus resulting in improved beta testing metrics.

According to the exemplary embodiments, beta testing results may bereceived (by, for example, application system 112) as feedback from betausers 134 and/or from analytics or metrics captured in the backgroundregarding how the particular beta user 134 uses a beta feature. In oneembodiment, as shown in FIGS. 9A and 9B, once a new beta feature isinstalled to client device 130, the native application enables beta user134 to rate the beta feature and provide specific comments concerningthe beta feature and its interaction with the native application. Forexample, a beta user 134 may be able to rate an installed beta featureby selecting a rate icon 902 within the Feature Center, as shown in FIG.9A. Alternatively, a rating icon or interface may be displayed uponaccessing the beta feature itself. Upon selecting rate icon 902, betauser 134 may be directed to a second screen 906, as shown in FIG. 9B,providing an interface for rating the beta feature. For example, a betauser 134 may be able to rate the “Refer a Friend” beta feature 836 on aparticular number or other scale such as a star scale shown in field908. Additionally, beta user 134 may be prompted to enter more detailedrating or review information such as via field 909. Upon completion ofits review, beta user 134 may transmit the rating and review informationto application system 112 such as by selecting a send icon 910. In oneembodiment, beta user 134 may be prompted to rate or review a particularbeta feature immediately after utilizing the beta feature or aparticular aspect of the beta feature. Additionally, in anotherembodiment, the native application may provide a rating section orrating icon for each particular aspect of a beta feature, such asregarding visual aesthetics, ease of use, ways to improve, desirabilityof the feature, etc. The rating section may be displayed on each screenor interface viewed within the beta feature or native application asdesired. In one embodiment, a beta user 134 may be encouraged to rate aparticular beta feature by incentivizing them with the prospect of beinggranted access to additional beta features, more favorable account terms(such as a higher interest rate on savings account, lower interest rateson credit cards, etc.), or other incentives.

In one embodiment, analytics and metrics may also be captured inreal-time (or substantially real-time) based on a beta user's 134 use ofa particular beta feature or standard feature, and automaticallyuploaded or sent to application system 112. For example, any number ofanalytics or metrics may be captured in real-time, such as the length oftime it takes a beta user 134 to execute a process, the number andfrequency of interactions with the beta feature, how often a beta user134 may request additional assistance from provider 110, when a betauser 134 exits the feature or the native application, and whetherinclusion of the beta feature results in increased interaction withother features of the native application, etc. The particular analyticsthat may be analyzed are limitless and are not limiting of the exemplaryembodiments. According to the exemplary embodiments, the gatheredanalytics may be compared to other non-beta users 132 and other betausers 134. The comparisons may then be analyzed in a number of ways asdesired. For example, the compared analytics may enable provider 110 orapplication system 112 to determine and report statistics indicatingwhether the beta features and functionality improve or increase userengagement with the native application based on, for example, the numberof interactions with a feature of a particular user pre- andpost-installation of the beta features, the number of user interactionswith one or more features between distinct beta and non-beta users, etc.

Additionally, application system 112 may prompt a beta user 134 toprovide additional information concerning its use of the particular betafeature or the native application if it identifies a particular patternof behavior. For example, if a beta user 134 frequently exits the nativeapplication without completing a process within the application or abeta feature, application system 112 may prompt beta user 134 to explainthose actions. Such prompt may be displayed to beta user 134 within thenative application itself, by e-mail, or any other suitable method.

Application system 112 may be capable of sorting the received ratingsand reviews and all other gathered analytics and metrics concerning abeta user's 134 use of a particular beta feature or the nativeapplication. Application system 112 may then use this information toimprove a beta feature before making it available to all other users132, if desirable. In some embodiments, a beta feature may be updatedand then provided to a new group of beta users 134 to test the updatedfunctionality of such beta feature. Each new beta feature may befrequently updated and provided to new beta users 134, if desired, untilthe beta feature is ready to implement to all other users 132 of thenative application. Such beta testing processes may enable applicationsystem 112 to rapidly implement compelling new features and improvecurrent features to maintain a positive customer experience with thenative application.

The above embodiments are not limited to any particular operating systemor platform. Each of the above embodiments may be realized byimplementing any software development kit available for any particularoperating system or platform. Although embodiments may be implemented ascomputer processing instructions, all or a portion of the functionalityof disclosed embodiments may be implemented instead in electronicshardware. Other embodiments will be apparent to those skilled in the artfrom consideration of the specification and practice of the disclosedembodiments. It is intended that the specification and examples beconsidered as exemplary only, with a true scope and spirit of thedisclosed embodiments being indicated by the following claims.Furthermore, although aspects of the disclosed embodiments are describedas being associated with data stored in memory and other tangiblecomputer-readable storage mediums, one skilled in the art willappreciate that these aspects can also be stored on and executed frommany types of tangible computer-readable media, such as secondarystorage devices, like hard disks, floppy disks, or CD-ROM, or otherforms of RAM or ROM. Accordingly, the disclosed embodiments are notlimited to the above described examples, but instead is defined by theappended claims in light of their full scope of equivalents.

What is claimed is:
 1. A system for identifying and enrolling aplurality of users into at least one test group, the system comprising:at least one memory device storing instructions; and at least oneprocessor in communication with at least one database, the at least oneprocessor being configured to execute the instructions to: provide anative application configured to be hosted at a client device, thenative application comprising a non-beta feature set and a beta featureset, the beta feature set comprising new beta features or updated betaversions of non-beta features of the beta feature set; enroll a firstuser of the plurality of users in a first test group and a second userof the plurality of users in a second test group, the at least one testgroup to test features of a native application executed by a clientdevice; wherein the first test group is associated with testing firstbeta features of the beta feature set, and the second test group isassociated with testing second beta features of the beta feature setthat are different from the first beta features; enable, based on theenrollment, the first user to access the first beta features such thatthe first user is enabled to access the non-beta feature set and thefirst beta feature via a first instance of the native applicationexecuting on a first client device of the first user; and enable, basedon the enrollment, the second user to access the second features suchthat the second user is enabled to access the non-beta feature set andthe second beta feature via a second instance of the native applicationexecuting on a second client device of the second user, wherein thefirst and second instances of the native application are a same versionof the native application.
 2. The system of claim 1 wherein the at leastone processor is further configured to store test group identificationinformation for the users.
 3. The system of claim 1 wherein the at leastone processor is further configured to send a communication signal tothe first client device including the test group identificationinformation for a user associated with the first client device.
 4. Thesystem of claim 1 wherein the test group identification informationincludes an indication that indicates whether the user is enrolled inthe first test group or the second test group.
 5. The system of claim 1wherein the test group identification information further includes anindication that indicates whether the user is a member or a non-memberof the first test group.
 6. The system of claim 1 wherein the at leastone processor is further configured to receive a request from the firstclient device to download the native application to the first clientdevice.
 7. The system of claim 1 wherein the at least one processor isfurther configured to dynamically update a current feature or add atleast one new feature to the first test group.
 8. A non-transitorycomputer-readable medium storing instructions executable by at least oneprocessor to perform operations for identifying and enrolling aplurality of users as users into at least one test group, the operationscomprising: providing a native application configured to be hosted at aclient device, the native application comprising a non-beta feature setand a beta feature set, the beta feature set comprising new betafeatures or updated beta versions of non-beta features of the betafeature set; enrolling a first user of, by the at least one processor,the plurality of users in a first test group and a second user of theplurality of users in a second test group, the at least one test groupto test features of a native application executed by a client device;wherein the a first test group is associated with testing first betafeatures of the beta feature set, and the second test group isassociated with testing second beta features of the beta feature setthat are different from the first beta features; and enabling, based onthe enrollment, first user to access the first beta features such thatthe first user is enabled to access the non-beta feature set and thefirst beta feature via a first instance of the native applicationexecuting on a first client device of the first user; and enabling,based on the enrollment, the second user to access the second featuressuch that the second user is enabled to access the non-beta feature setand the second beta feature via a second instance of the nativeapplication executing on a second client device of the second user,wherein the first and second instances of the native application are asame version of the native application.
 9. The non-transitorycomputer-readable medium of claim 8, the operations further comprising,storing test group identification information for the users.
 10. Thenon-transitory computer-readable medium of claim 8, the operationsfurther comprising, sending a communication signal to the first clientdevice including the test group identification information for a userassociated with the first client device.
 11. A method for identifyingand enrolling a plurality of users into at least one test group, themethod comprising: storing feature information indicating test featuresof the application, the test features being configured in theapplication as features to be tested by users enrolled in a test group;enrolling a first user in a first test group and a second user in asecond test group, wherein the first test group is associated withtesting at least a first test feature of the test features, and thesecond test group is associated with testing at least a second testfeature of the test features that is different from the first testfeature; enabling, based on the enrollment, the first user to access thefirst test feature such that the first user is enabled to access thefirst test feature via a first instance of the application executing ona first client device of the first user; and enabling, based on theenrollment, the second user to access the second test feature such thatthe second user is enabled to access the second test feature via asecond instance of the application executing on a second client deviceof the second user.
 12. The method of claim 11, further comprisingstoring test group identification information for the users.
 13. Themethod of claim 12, further comprising sending a communication signal tothe first client device including the test group identificationinformation for a user associated with the first client device.
 14. Thesystem of claim 12, wherein the test group identification informationincludes an indication that indicates whether the user is enrolled inthe first test group or the second test group.
 15. The system of claim12, wherein the test group identification information further includesan indication that indicates whether the user is a member or anon-member of the first test group.
 16. The method of claim 11, furthercomprising receiving a request from the first client device to downloadthe native application to the first client device.
 17. The method ofclaim 11, further comprising dynamically updating a current feature oradd at least one new feature to the first test group.